System and method for processing wireless backscattered signal using artificial intelligence processing for activities of daily life

ABSTRACT

In an example, the technique also detects and measures vital signs of each human target by continuous, non-intrusive method. In an example, the vital signs of interest include a heart rate and a respiratory rate, which can provide valuable information about the human&#39;s wellness. Additionally, the heart rate and respiratory rate can also be used to identify a particular person, if more than two target humans living in a home. Of course, there can be other variations, modifications, and alternatives.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional application62/545,921 filed Aug. 15, 2017, which is incorporated by referenceherein for all purposes.

BACKGROUND

The present invention relates to techniques, including a method, andsystem, for processing signals using artificial intelligence techniquesto monitor, detect, and act on activities. In an example, the signalscan be from both active and passive sensors, among others. Merely by wayof examples, various applications can include daily life, and others.

Various conventional techniques exist for monitoring people within ahome or building environment. Such techniques include use of cameras toview a person. Other techniques include a pendant or other sensingdevice that is placed on the person to monitor his/her movement.Examples include Personal Emergency Response Systems (PERS) devices suchas LifeAlert® and Philips® LifeLine—each of which are just panic buttonsfor seniors to press in case of an emergency. Unfortunately, all ofthese techniques have limitations. That is, each of these techniquesfails to provide a reliable and high quality signal to accurately detecta fall or other life activity of the person being monitored. Many peopleoften forget to wear the pendant or a power source for the pendant runsout. Also, elderly people do not want to look like they are old so oftentimes, elderly people do not wear the pendant.

From the above, it is seen that techniques for identifying andmonitoring a person is highly desirable.

SUMMARY

According to the present invention, techniques, including a method, andsystem, for processing signals using artificial intelligence techniquesto monitor, detect, and act on activities are provided. In an example,the signals can be from both active and passive sensors, among others.Merely by way of examples, various applications can include daily life,and others.

In an example, the present invention provides a sensor array in a singlebox that can be placed in a home or a single box (acting as a basestation) that talks to multiple helper sensor boxes distributedthroughout a living space of the home. In an example, the sensor arraywill communicate with a backend server via standard connectivitysolutions, such as Wi-Fi, cellular, or others. In an example, thetechnique uses distributed processing where processing of the dataoccurs inside the sensor array and in a cloud server. In an example,artificial intelligence (AI) techniques are included. Depending upon theexample, the processed data are disseminated to various interestedparties (e.g., children of elderly person, care takers, EmergencyMedical Response team) via different communication channels, such assmartphone app, SMS, email, voicemail, and other techniques.

In an example, the present invention provides a method of detecting astatus of a human being or target. The method includes transferring,using a wireless transmitter, a wireless signal being selected from oneor more of a frequency being less than about 10 G Hz, 24 G Hz, 60 G Hz,or 77 G Hz and greater. The method includes capturing a back scatteredsignal, using an rf antenna, from the wireless signal. The methodincludes processing the back scattered signal to extract one or more ofa direction, signal strength, distance, and other information over atime period. The method includes extracting, using a signal processingprocess, vital signs of a human, the vital signs including a heart rate,or a respiration rate. The method includes creating a baseline for eachof the vital signs. The method includes extracting, using an AI process,a physical activity of the human being. The method includes creating aphysical activity base line for the physical activity and determining aconfidence level of each of the received vital signals and each of thephysical activities. The method includes transferring an alert toanother target upon a triggering even based upon the confidence level ofeach of the received vital signals and each of the physical activitiesand correlating each vital sign, using an artificial intelligenceprocess, with one or more patterns or the base line for each of thevital signs.

In an example, the present invention provides a system for monitoringand detecting an activity of a human target. The system has a sensorarray, the sensor array comprising a plurality of passive sensors. In anexample, each of the plurality of passive sensors is spatially disposedin spatial region of a living area. In an example, the system has awireless backscattering detection system. The wireless backscatteringdetection system has a control line coupled to a processing device. Inan example, the control line is configured with a switch to trigger aninitiation of a wireless signal. The detection system has a waveformpattern generator coupled to the control line, an rf transmitter coupledto the waveform pattern generator, a transmitting antenna coupled to therf transmitter, an rf receiver, an rf receiving antenna coupled to therf receiver, an analog front end comprising a filter, an analog todigital converter coupled to the analog front end, a signal processingdevice coupled to the analog to digital converter, and an artificialintelligence module coupled to the signal processing device, andconfigured to process information associated with a backscattered signalcaptured from the rf receiving antenna. Further details of each of theseelements can be found throughout the present specification and moreparticularly below.

The above examples and implementations are not necessarily inclusive orexclusive of each other and may be combined in any manner that isnon-conflicting and otherwise possible, whether they be presented inassociation with a same, or a different, embodiment or example orimplementation. The description of one embodiment or implementation isnot intended to be limiting with respect to other embodiments and/orimplementations. Also, any one or more function, step, operation, ortechnique described elsewhere in this specification may, in alternativeimplementations, be combined with any one or more function, step,operation, or technique described in the summary. Thus, the aboveexamples implementations are illustrative, rather than limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified diagram of a radar/wireless backscattering sensorsystem according to an example of the present invention.

FIG. 2 is a simplified diagram of a sensor array according to an exampleof the present invention.

FIG. 3 is a simplified diagram of a system according to an example ofthe present invention.

FIG. 4 is a detailed diagram of hardware apparatus according to anexample of the present invention.

FIG. 5 is a simplified diagram of a hub according to an example of thepresent invention.

FIG. 6 is a simplified diagram of a hub in a spatial region according toan example of the present invention.

FIG. 7 is a simplified diagram of a mini node according to an example ofthe present invention.

FIG. 8 is a simplified diagram of a mini mode in a spatial regionaccording to an example of the present invention.

FIG. 9 is a simplified diagram of a mobile node according to an exampleof the present invention.

FIG. 10 is a simplified diagram of a mobile mode in a spatial regionaccording to an example of the present invention.

FIG. 11 is a simplified diagram illustrating senor ADL categories in anexample.

FIG. 12 is a simplified diagram illustrating an activity list accordingto an example.

DETAILED DESCRIPTION OF THE EXAMPLES

According to the present invention, techniques, including a method, andsystem, for processing signals using artificial intelligence techniquesto monitor, detect, and act on activities are provided. In an example,the signals can be from both active and passive sensors, among others.Merely by way of examples, various applications can include daily life,and others.

FIG. 1 is a simplified diagram of a radar/wireless backscattering sensorsystem 100 according to an example of the present invention. Thisdiagram is merely an example, which should not unduly limit the scope ofthe claims herein. In an example, the system is a wirelessbackscattering detection system. The system has a control line 110coupled to a processing device 120, the control line being configuredwith a switch to trigger an initiation of a wireless signal. In anexample, the system has a waveform pattern generator 130 coupled to thecontrol line 110. The system has an rf transmitter 140 coupled to thewaveform pattern generator 130. The system has transmitting 150 andreceiving 160 antennas. In an example, the system has a transmittingantenna 150 coupled to the rf transmitter 140 and an rf receiver 140,which is coupled to an rf receiving antenna 160. In an example, thesystem has an analog front end 170 comprising a filter. An analog todigital converter 180 coupled to the analog front end 170. The systemhas a signal-processing device 120 coupled to the analog to digitalconverter 180. In a preferred example, the system has an artificialintelligence module coupled to the signal-processing device 120. Themodule is configured to process information associated with abackscattered signal captured from the rf receiving antenna. Furtherdetails of the present system can be found through out the specificationand more particularly below.

Antenna

In an example, multiple aspects of antenna design can improve theperformance of the activities of daily life (“ADL”) system. For example,in scanning mode the present technique continuously looks for movinghuman targets (or user) to extract ADL or fall. Since these can happenanywhere in the spatial region of a home, the present system hasantennas that have wide field of view. Once the human target isidentified, the technique focuses signals coming only from thatparticular target and attenuate returns from all other targets. This canbe done by first estimating location of the target from our techniqueusing wide field of view antennas and then focusing RF energy on thespecific target of interest once it has been identified. In an example,the technique can either electronically switch a different antenna thathas narrow field of view or could use beam forming techniques tosimultaneously transmit waves from multiple transmit antenna and controltheir phase such that the RF energy constructively builds around thetarget of interest where as it destructively cancels everywhere else.This return will be much cleaner and can boost the performance of ourADL+fall+vital sign sensors.

In another example considers the layout of the antennas itself. In anexample, the technique places transmit and receive antennas in variousphysical configurations (ULA, circular, square, etc.), that can help usestablish the direction from which the radar signal returns, bycomparing phases of the same radar signal at different receivingantennas. The configurations can play a role because differentconfigurations enable direction of arrival measurement from differentdimensions. For example, when the human target falls the vertical angleof arrival changes from top to bottom, therefore a vertical ULA isbetter suited to capture that information. Likewise during walkinghorizontal angle of arrival of the signal varies more therefore it makessense to use horizontal ULA is more sensitive and therefor can provideadditional information for our algorithm. Of course, there can be othervariations, modifications, and alternatives.

RF Unit

In an example, the wireless RF unit can be either pulsed doppler radaror frequency modulated continuous wave (FMCW) or continuous wave doppler(CW). In an example, on the transmit side it will have standard RF unitslike VCO, PLL, among others. On the receive side it can have matchedfilter, LNA, mixer, and other elements. The multiple antennas can beeither driven by a single transmit/receive chain by sharing it in timeor have one each chain for each of the antennas.

Waveform Unit

In an example, waveform pattern generator generates control signals thatdefine the type of radar signal that is generated by the radar RF unit.For example, for FMCW, it can generate triangular wave of specific slopeand period, which will linearly sweep the frequency of the RF unitaccording to this parameter. For a pulsed doppler radar, the techniquewill hold generate pulse of specific width and period, which willmodulate the RF output accordingly.

Baseband Unit

In an example, the gain and filter stage filters the radar returns toremove any unwanted signals and then amplifies the remaining signal withdifferent techniques. For example, the present artificial intelligenceor AI technique can determine what target is desirably tracked andprovide feedback to the AI technique, that will filter out radar returnfrom any and all other signals except for the signal that is desirablytracked. If human target is moving the return signal will befluctuating, in that case, the technique applies automatic gain control(AGC) to find the optimal gain, so that entire dynamic range of ADC inthe subsequent stage is satisfied. In an example, the return signal isconverted to digital samples by analog-to-digital converters (ADC),among other front-end elements.

FIG. 2 is a simplified diagram of a sensor array 200 according to anexample of the present invention. This diagram is merely an example,which should not unduly limit the scope of the claims herein. Shown is asensor array. The sensor array includes a plurality of passive sensors210. In an example, the plurality of passive sensors are spatiallydisposed in spatial region of a living area. The sensor array has activesensors, such as one or more radar sensors 220. Additionally, the arrayhas a feedback interface, such as a speaker 230 for calling out to ahuman target in the spatial region of the living area.

In an example, the present technique is provided to identify variousactivities in home using non-wearable. In an example, the technique isat least privacy intrusive as possible, and will use sensors that areless intrusive. Examples of sensors can include, without limitation, awireless backscatter (e.g., radar, Wi-Fi.), audio (e.g., microphonearray, speaker array), video (e.g., PTZ mounted, stereo), pressure mats,infrared, temperature, ultraviolet, humidity, pressure, smoke, anycombination thereof, and others.

Active Sensor for RADAR

In an example, the technique can use wireless backscattering to measuremotion of human, a location, and an environmental state, such as dooropening/closing, or other environmental condition. In an example, thewireless backscattering can also be used to measure a vital sign, suchas a heart rate and respiration rate, among others. In an example, thewireless techniques can work in non-line of sight, and is non intrusivecompared to camera or microphone, or others. In an example, thetechnique can use radar\backscatter sensor for two purposes (1) to findthe location of an action; and (2) sense different activities associatedwith the action. Of course, there can be other variations,modifications, and alternatives.

In an example, the present technique and system includes a radar systemthat operates on multiple frequency bands, such as below 10 GHz, around24 GHz, 60 GHz, 77-81 GHz, among others. In an example, differentfrequency interacts differently with various objects in our environment.In an example, available signal bandwidth and permissible signal powerare also regulated differently at different frequency bands. In anexample, the present techniques optimally combine reflections comingfrom a reflector from multiple frequency bands to achieve largecoverage, and/or improve accuracy. Of course, there can be othervariations, modifications, and alternatives.

In an example, each radar is working at a particular frequency band willbe using multiple transmit and receive antennas, as shown. In anexample, using these multiple transmitters, the technique can performtransmit beam forming to concentrate radar signal on a particulartarget. In an example, the technique uses multiple receivers to collectreflected signals coming from various reflectors (e.g., human body,walls). After further processing this will allow us to find thedirection of the reflector with respect to the radar. In an example, thetechnique also uses multiple transmitter and receiver to form virtualarray, this will allow emulate the radar array with large element byusing small number of transmitter and receiver chains. The main benefitis to improve the angle resolution without using a large array, savingspace and component cost. In an example, different antenna arrayconfigurations to improve coverage (using beam forming) or add 3Dlocalization capability (using 2-D array) are included.

In an example using standard radar signal modulation techniques, such asFMCW/UWB, on MIMO radar, the technique will first separate signalscoming from different range and angle. The technique will then identifystatic reflectors, such as chairs, walls, or other features, from movingones, such as human targets, pets, or the like. For moving objects thatare tracked, the technique will further process signals for each of thereflectors. As an example, the technique will use different techniquesto extract raw motion data (e.g., like spectrogram). In an example, thetechnique will apply various filtering process to extract periodicsignals generated by vital signs, such as heart rate, respiration rate,among others. In an example, both the raw motion data and extractedvital signs will be passed to a downstream process, where they arecombined with data from other sensors, such as radar outputs operatingat different frequency or completely different sensors to extract higherinsights about the environment. Of course, there can be othervariations, modifications, and alternatives.

Audio Sensor

In an example, the present technique uses a sensor array that has amultiple microphone array 240. In an example, these microphones 240 willbe use to ascertain the direction of arrival of any audio signal in theenvironment. In an example, the microphone 240 in conjunction with othersensors, such as radar 220, will be vital in performing two tasks: 1stit will augment radar signal to identify various activities (walkingproduces a different sound than sitting), if the target is watching TVit is much easier to ascertain it with audio signal; and 2nd in case ofemergency like fall, the technique can use the radar signal to identifythe location of the fall and then beam form microphone array towardsthat location, so that any audio signal produced by the target can becaptured. Of course, there can be other variations, modifications, andalternatives.

Sensor Fusion and Soft Sensors

In addition to a radar sensor, which is consider as active sensors thepresent sensor system (e.g., box, boxes) will also have additionalpassive sensors that captures the sound, chemical signature,environmental conditions. Each of these of the sensors capturesdifferent context about the home that the human being tracking is livingin or occupying. In an example, the UV 250 sensor can monitor how oftenthe sunlight comes in the room. In an example, light sensors determine alighting condition of the human's home or living area.

In an example, a microphone array 240 can have many functions, such asuse to sense sound in the room, to figure out how long the human hasspent watching TV, or how many time they went to bathroom by listeningto the sound of toilet flushing or other audio signature. In an example,the present technique can use creative solutions where it can use theactive sensor to find the location of the person and then tune themicrophone array to enhance the sound coming from that location only,among other features. In an example, the technique can call the sensorsthat are derived from the hardware sensors using specific algorithms assoftware sensors or soft sensors. So the same hardware sensors can beused for many different applications by creating different softwaresensors. Here the software sensors can combine signals from one or moresensors and then apply sensor fusion and Al techniques to generate thedesired output. Of course, there can be other variations, modifications,and alternatives.

Soft Sensor for Detecting Cooking and Eating Habits

In example, radar sensors can determine information about a human'slocation within a home, like if they are in kitchen area, or other. Inan example, when the human target turns on the microphone oven, itgenerates specific RF signature that can be tracked. In an example, thetechnique can combine this information to infer if the human targetwalked to the kitchen and turned on the microphone. Likewise, when thehuman target prepares food in kitchen he/she can make lot of specificnoise like utensils clattering, chopping, or other audio signature. Soif a human target goes to kitchen spends sometime time in the kitchen,and the present microphone pick these sounds, the technique can inferthat food is cooking or other activity.

Soft Sensor for Detecting Bathroom Habits

In an example, toileting frequency can be a very valuable indication ofones wellness. The present technique can track if a human went to thebathroom using the radar or other sensing techniques. In an example,additionally, the technique can pick sound signature of toilet flushing.In an example, the technique combines these two pieces of information,which can be correlated to toileting frequency. In an example,similarly, bathing is a unique activity that requires 4-5 minutes ofspecific movements. By learning those patterns, the technique can figureout ones bathing routines.

Soft Sensor for Detecting Mobile Habits

In an example, different sensors are triggered by different motion of ahuman target. In an example, radar can detect human fall by looking atmicro doppler patterns generating by different part of the target duringfalls. In an example, the technique can also simultaneously hear a fallfrom microphone arrays and vibration sensors. In an example, thetechnique can also detect how pace of movement changes for an individualover a long duration by monitoring the location information provided byradar or other sensing technique. In an example, likewise, the techniquecan gather unstable transfers by analyzing the gait of the target. In anexample, the technique can find front door loitering by analyzing theradar signal pattern. In an example, the technique can figure outimmobility by analyzing the radar return. In this case, the techniquecan figure out the target's presence by analyzing the target's vitalsigns, such as respiration rate or heart rate or by keeping track of thebread crumb of the target's location trace.

In any and all of the above cases, the technique can also learn aboutthe exact environmental condition that triggered a particular state. Forexample, the technique can figure out whether a human target wasimmobile because the target was watching TV or a video for long durationor the target was simply spending a lot of time in their bed. And thesecan be used to devise incentives to change the target's behavioralpattern for better living.

Soft Sensor for Detecting Vital Signs

In an example, the technique can estimate vital signs of a person bysensing the vibration of the target's body in response to the breathingor heart beat, each of the actions results in tiny phase change in theradar return signals, which can be detected. In an example, thetechnique will use several signal processing techniques to extract them.Of course, there can be other variations, modifications, andalternatives.

In an example, different frequency radio wave interact with environmentdifferently. Also phase change due to vital signs (HR,RR) differs byfrequency, for example phase change for a 77 GHz radar is much higherthan for a 10 GHz radar. Thus 77 GHz is more appropriate for estimatingheart-beat more accurately. But higher frequency typically attenuatesmuch more rapidly with distance. Therefore, lower frequency radar canhave much larger range. By using multi-frequency radar in the presenttechnique can perform these vital trade-offs.

Soft Sensor for Detecting Sleeping Habits

In an example, the present radar sensors can detect motions that aregenerated during sleep, such as tossing and turning. In an example,radar sensors can also sense vital signs like respiration rate and heartrate as described earlier. In an example, now combining the pattern oftoss and turn and different breathing and heart beat pattern, thetechnique can effectively monitor the target's sleep. Additionally, thetechnique can now combine results from passive sensors, such as athermometer, UV, photo diode, among others, to find correlation betweencertain sleep pattern and the environmental conditions. In an example,the technique can also use the sleep monitor soft sensor to learn aboutday/night reversal of sleep, and the associated environmental conditionby looking at different passive sensors. In an example, the techniquescan be valuable in providing feedback to improve the human target'ssleep. For example, the technique can determine or learn that certainenvironmental condition results in better sleep and prescribe that toimprove future sleep.

Soft Sensor for Security Applications

In an example, the technique can repurpose many of the sensors describedbefore for security applications. For a security application, thetechnique determines where one or more person is located, which can bedetected using a presence detection sensor that is build on top of radarsignals. In an example, the technique can eliminate one or many falsepositive triggered by traditional security systems. For example, is awindow is suddenly opened by a wind the technique (and system) will lookat presence of human in the vicinity before triggering the alarm.Likewise, combination of vital signs, movement patterns, among others,can be used a biometric to identify any human target. If an unknownhuman target is detected in the vicinity at certain time of the day, thetechnique can trigger an alarm or alert.

In an example, any one of the above sensing techniques can be combined,separated, or integrated. In an example, n addition to radar and audiosensors, other sensors can be provided in the sensor array. Of course,there can be other variations, modifications, and alternatives.

FIG. 3 is a simplified diagram of a system according to an example ofthe present invention. This diagram is merely an example, which shouldnot unduly limit the scope of the claims herein. As shown, the systemhas hardware and method (e.g., algorithm), cloud computing, personalizedanalytics, customer engagement, and an API to various partners, such aspolice, medical, and others. Further details of the present system canbe found throughout the present specification and more particularlybelow.

FIG. 4 is a detailed diagram of hardware apparatus according to anexample of the present invention. This diagram is merely an example,which should not unduly limit the scope of the claims herein. As shown,the hardware units include at least a hub device, node, and mobile node,each of which will be described in more detail below.

FIG. 5 is a simplified diagram of a hub according to an example of thepresent invention. This diagram is merely an example, which should notunduly limit the scope of the claims herein. In an example, the hubincludes various sensing devices. The sensing devices, include, amongothers, a radar, a WiFi, a Bluetooth, a Zigbee sniffer, a microphone andspeakers, a smoke detector, a temperature detector, a humidity detector,a UV detector, a pressure detector, MEMS (e.g., accelerometer,gyroscope, and compass), a UWB sensors (for finding locations of all thedeployed elements relative to each other), among others. In an example,the hub is a gateway to internet via Wi-Fi, GSM, Ethernet, landline, orother technique. The hub also connects to other units (Mini Node/MobileNode) via Bluetooth, Wi-Fi, Zigbee, UWB and coordinates them with eachother. In an example, certain data processing, such as noise removal,feature extraction to reduce amount of data uploaded to cloud isincluded. In an example, the hub alone can be sufficient to cover asmall living space. In an example, the hub is deployed as a singledevice somewhere in a desirable location (e.g., middle of the livingspace) so that it has good connectivity to all other units. An exampleof such deployment is provided in the Figure below.

FIG. 6 is a simplified diagram of a hub in a spatial region according toan example of the present invention. This diagram is merely an example,which should not unduly limit the scope of the claims herein. As shown,the hub is deployed in the middle of the living space in a house.

FIG. 7 is a simplified diagram of a mini node according to an example ofthe present invention. This diagram is merely an example, which shouldnot unduly limit the scope of the claims herein. As shown, the systemhas sensors, which is a subset of sensors in the hub. The sensors areconfigured to in various spatial locations to improve coverage area andimprove accuracy for detection of critical events (e.g., fall, someonecalling for help). The sensors also communicate with the hub via Wi-Fi,Bluetooth, ZigBee or UWB, or other technique. Additionally, the sensorsor each mini node is deployed in a bathrooms, where chances of fall ishigh, a kitchen, where we can learn about eating habits by listening tosounds, RF waves, vibrations, or a perimeter of the living space, thatwill allow us to learn approximate map of the space under consideration,among other locations. Additionally, each of the mini modes can savepower and costs by adding more complexity on the hub. This can evenenable us to operate on battery for extended periods. For example, eachof the nodes can have only single antenna Wi-Fi and hub could havemultiple antennas, for WiFi based sensing. Additionally, each of thenodes use simpler radar (e.g., single antenna doppler) vs MIMO FMCW inthe HUB. Additionally, each node can be configured with a singlemicrophone whereas the hub can have array of microphone. Of course,there can be other variations, modifications, and alternatives.

FIG. 8 is a simplified diagram of a mini mode in a spatial regionaccording to an example of the present invention. This diagram is merelyan example, which should not unduly limit the scope of the claimsherein. As shown, each node is configured in a kitchen, shower,perimeter, or other location.

FIG. 9 is a simplified diagram of a mobile node according to an exampleof the present invention. This diagram is merely an example, whichshould not unduly limit the scope of the claims herein. In an example,each mobile node is a subset of sensors in the hub. The mobile nodesensors include a camera such as RGB or IR. In an example, each of thenodes and hub collaboratively figure out interesting events, and passthat information to the mobile node. The technique then goes to thelocation and probes further. In an example, the camera can be useful tovisually find what is going on in the location. In an example, freewillpatrolling can be use to detect anything unusual or to refine details ofthe map created based on perimeter nodes. In an example, onboard UWB canenable precise localization of the mobile node, which can also enablewireless tomography, where the precise RGB and wireless map of theliving space is determined.

FIG. 10 is a simplified diagram of a mobile mode in a spatial regionaccording to an example of the present invention. This diagram is merelyan example, which should not unduly limit the scope of the claimsherein. As show, the mobile node, such as a mobile phone or smart phoneor other movable device, can physically move throughout the spatiallocation. The mobile node can also be a drone or other device.

In an example, the technique transfers learned information and activityinformation to third parties. The technique teaches itself to learn highlevel behavior that are indicative of a person's welfare usingartificial intelligence techniques. In an example, the present techniquewill then generate summary of such activities and send it out to thehuman's loved ones, caretaker or even emergency response team dependingon the urgency of the situation. For example, for regular days, thetechnique can simply send short summary like “your mom had a routineactivity today”, or “She was much less active today.” In an example,where the human has a care taker visiting few times a week, thetechnique can send a notification to them, “It seems she struggles moreon yesterday”, so that the care taker can pay a visit to make sureeverything is fine. Alternatively, the technique can be more acuteevents like fall, shortness of breathing, or others, that needs quickattention. In these scenarios, the technique can notify medical responseteam to provide immediate help. Of course, there can be othervariations, modifications, and alternatives.

FIG. 11 is a simplified diagram illustrating senor ADL categories in anexample. As shown, the present technique can categorize a human targetwith the listed ADLs, among others.

FIG. 12 is a simplified diagram illustrating an activity list accordingto an example. As shown, the present technique can determine activitiesof a human target with any one of the activities listed.

In an example, the present technique can also identify a rare event. Inan example, the technique identifies when a senior human falls inside ahome with no one around. In an example, the technique is robust, withoutany false negatives. In an example, the technique uses looking atsequence of events that are before to the potential fall and after apotential fall. In an example, the technique combines the contextualinformation to robustly determine if a fall has occurred. Of course,there can be other variations, modifications, and alternatives.

In an example, the technique also detects and measures vital signs ofeach human target by continuous, non-intrusive method. In an example,the vital signs of interest include a heart rate and a respiratory rate,which can provide valuable information about the human's wellness.Additionally, the heart rate and respiratory rate can also be used toidentify a particular person, if more than two target humans living in ahome. Of course, there can be other variations, modifications, andalternatives.

By understanding the context of how the target human (e.g., elderly) isdoing, the technique can also provide valuable feedback directly to theelderly using a voice interface. For example, the technique can sense amood of the human based on sequence of activities and vital signs of thehuman and then ask, “Hi do you want me to call your son”. Based upon thefeedback from the human, the technique can help connect to a third party(or loved one) if their answer is positive. Of course, there can beother alternatives, variations, and modifications.

Having described various embodiments, examples, and implementations, itshould be apparent to those skilled in the relevant art that theforegoing is illustrative only and not limiting, having been presentedby way of example only. Many other schemes for distributing functionsamong the various functional elements of the illustrated embodiment orexample are possible. The functions of any element may be carried out invarious ways in alternative embodiments or examples.

Also, the functions of several elements may, in alternative embodimentsor examples, be carried out by fewer, or a single, element. Similarly,in some embodiments, any functional element may perform fewer, ordifferent, operations than those described with respect to theillustrated embodiment or example. Also, functional elements shown asdistinct for purposes of illustration may be incorporated within otherfunctional elements in a particular implementation. Also, the sequencingof functions or portions of functions generally may be altered. Certainfunctional elements, files, data structures, and so one may be describedin the illustrated embodiments as located in system memory of aparticular or hub. In other embodiments, however, they may be locatedon, or distributed across, systems or other platforms that areco-located and/or remote from each other. For example, any one or moreof data files or data structures described as co-located on and “local”to a server or other computer may be located in a computer system orsystems remote from the server. In addition, it will be understood bythose skilled in the relevant art that control and data flows betweenand among functional elements and various data structures may vary inmany ways from the control and data flows described above or indocuments incorporated by reference herein. More particularly,intermediary functional elements may direct control or data flows, andthe functions of various elements may be combined, divided, or otherwiserearranged to allow parallel processing or for other reasons. Also,intermediate data structures of files may be used and various describeddata structures of files may be combined or otherwise arranged.

In other examples, combinations or sub-combinations of the abovedisclosed invention can be advantageously made. Some embodiments mayincorporate smart speaker interface and controls, such as currentlyprovided by Google Home, Amazon Alexa, Apple HomePod and others. Forexample, using the sensor and Al techniques described above, the devicemay perform appropriate actions. As examples of this, if the devicedetermines that the user has fallen down and cannot get up, the devicemay call for help, turn on all the lights, and/or unlock the doors; ifthe device determines that the user is cooking, the device may turn onan exhaust fan, increase sensitivity for a smoke detector, and/or turnon the lights in the kitchen; if the device determines that the user isalone watching television, the device may turn off lights in otherrooms; turn down the light in the room the user is in; and turn offmusic playing in other rooms; and the like. In light of the presentdisclosure, one of ordinary skill in the art should recognize many othertypes of actions that may be performed based upon the user sensedactivity.

The block diagrams of the architecture and flow charts are grouped forease of understanding. However it should be understood that combinationsof blocks, additions of new blocks, re-arrangement of blocks, and thelike are contemplated in alternative embodiments of the presentinvention. Further examples of embodiments of the present invention areprovided in the attached appendix.

Examples of processing techniques can be found in Exhibit 1, which isincorporated by reference herein.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

1. A method of detecting a status of a human being, the methodcomprising: transferring, using a wireless transmitter, a wirelesssignal being selected from one or more of a frequency being less thanabout 10 G Hz, 24 G Hz, 60 G Hz, or 77 G Hz and greater; capturing aback scattered signal from the wireless signal; processing the backscattered signal to extract one or more of a direction, signal strength,distance, and other information over a time period; extracting, using asignal processing process, received vital signs of a human, the vitalsigns including a heart rate, or a respiration rate; creating a baselinefor each of the received vital signs; extracting, using an AI process, aphysical activity of the human being; creating a physical activity baseline for the physical activity; determining a confidence level of eachof the received vital signals and each of the physical activities;transferring an alert to another target upon a triggering even basedupon the confidence level of each of the received vital signals and eachof the physical activities; and correlating each received vital sign,using an artificial intelligence process, with one or more patterns orthe base line for each of the received vital signs.
 2. The method ofclaim 1 further comprising initiating a process to transfer inquiryinformation to the human based upon the alert, the process including avoice initiated process to ask the human about the alert.
 3. The methodof claim 1 further comprising: capturing a captured signal associatedwith a MEMS device; and processing the captured signal.
 4. The method ofclaim 1 further comprising capturing a signal associated with anelectromagnetic wave associated with a frequency ranging from 2.4 GHz to6 GHz.
 5. The method of claim 1 further comprising capturing a signalassociated with a sensor selected from a group consisting of: anchemical sensor and a radio frequency sensor.
 6. The method of claim 1further comprising capturing a signal with a passive sensor selectedfrom a group consisting of: a microphone array, a photo diode, a UVsensor, a carbon monoxide sensor, smoke detector, and a pollen sensor.7. The method of claim 1 further comprising capturing a signal with apassive sensor selected from a group consisting of: a temperaturesensor, a microwave sensor, a humidity sensor, a vibration sensor, anacceleration sensor, a rotation sensor, a magnetic sensor.
 8. A systemfor monitoring and detecting an activity of a human target, the systemcomprising: a sensor array, the sensor array comprising: a plurality ofpassive sensors, the plurality of passive sensors being spatiallydisposed in spatial region of a living area; a wireless backscatteringdetection system comprising: a control line coupled to a processingdevice, the control line being configured with a switch to trigger aninitiation of a wireless signal; a waveform pattern generator coupled tothe control line; an rf transmitter coupled to the waveform patterngenerator; a transmitting antenna coupled to the rf transmitter; an rfreceiver; an rf receiving antenna coupled to the rf receiver; an analogfront end comprising a filter; an analog to digital converter coupled tothe analog front end; a signal processing device coupled to the analogto digital converter; an artificial intelligence module coupled to thesignal processing device, and configured to process informationassociated with a backscattered signal captured from the rf receivingantenna.
 9. The system of claim 8 wherein the plurality of passivesensors are selected from a group consisting of: a microphone array, aphoto diode, a UV sensor, a carbon monoxide sensor, smoke detector and apollen sensor.
 10. The system of claim 8 wherein the plurality ofpassive sensors are selected from a group consisting of: a temperaturesensor, a microwave sensor, a humidity sensor, a vibration sensor, anacceleration sensor, a rotation sensor, and a magnetic sensor.
 11. Thesystem of claim 8 further comprising a feedback interface comprising aspeaker to communicate with the human target.
 12. The system of claim 8wherein the rf transmitter emits an rf signal having an rf frequency of10 GzHz and below, around 24 GHz, and 77 GHz and above.
 13. The systemof claim 8 wherein the information is associated with a status factorselected from a group consisting of: motion of the human target, alocation of the human target, an environmental status, an open door, aclosed door, an open window, or other environmental characteristic. 14.The system of claim 13 wherein the information is selected from a groupconsisting of: a vital sign, a heart rate, a respiration rate, alocation of the human target, and an activity of the human target.
 15. Asystem for monitoring and detecting an activity of a human target, thesystem comprising: a hub comprising a sensor array, the sensor arraycomprising: a plurality of passive sensors, the plurality of passivesensors being spatially disposed in spatial region of a living area; awireless backscattering detection system comprising: a control linecoupled to a processing device, the control line being configured with aswitch to trigger an initiation of a wireless signal; a waveform patterngenerator coupled to the control line; an rf transmitter coupled to thewaveform pattern generator; a transmitting antenna coupled to the rftransmitter; an rf receiver; an rf receiving antenna coupled to the rfreceiver; an analog front end comprising a filter; an analog to digitalconverter coupled to the analog front end; a signal processing devicecoupled to the analog to digital converter; an artificial intelligencemodule coupled to the signal processing device, and configured toprocess information associated with a backscattered signal captured fromthe rf receiving antenna; a mobile node coupled to the hub andconfigured to communicate to the hub; a plurality of mini nodes coupledto the hub and each of the mini nodes configured to communicate to thehub.
 16. The system of claim 15 wherein the plurality of passive sensorsare selected from a group consisting of: a microphone array, a photodiode, a UV sensor, a carbon monoxide sensor, smoke detector and apollen sensor.
 17. The system of claim 15 wherein the plurality ofpassive sensors are selected from a group consisting of: a temperaturesensor, a microwave sensor, a humidity sensor, a vibration sensor, anacceleration sensor, a rotation sensor, and a magnetic sensor.
 18. Thesystem of claim 15 further comprising a feedback interface comprising aspeaker to communicate to the human target.
 19. The system of claim 15wherein the rf transmitter emits an rf signal having an rf frequency of10 GzHz and below, around 24 GHz, and 77 GHz and above.
 20. The systemof claim 15 wherein the information is associated with an activityselected from a group consisting of: motion of the human target, alocation of the human target, an environmental status, an open door, aclosed door, an open window, a heart rate or a respiration rate.
 21. Thesystem of claim 15 wherein the transmitting antenna comprises aplurality of rf transmitting antenna, and the rf receiving antennacomprises a plurality of rf receiving antenna.